Papers with demonstration selection
UniICL: An Efficient ICL Framework Unifying Compression, Selection, and Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve reasoning abilities of Large Language Models (LLMs) have limitations due to excessive growth in context length, causing large hardware burden. |
| Approach: | They propose a novel Unified ICL framework that unifies demonstration compression, demonstration selection, and final response generation. |
| Outcome: | The proposed framework unifies demonstration compression, demonstration selection, and final response generation. |
The Whole is Better than the Sum: Using Aggregated Demonstrations in In-Context Learning for Sequential Recommendation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown excellent performance on various NLP tasks. |
| Approach: | They propose a method that integrates multiple demonstration users into one aggregated demonstration to improve sequential recommendation. |
| Outcome: | The proposed method outperforms state-of-the-art LLM-based sequential recommendation methods on three recommendation datasets. |
D.Va: Validate Your Demonstration First Before You Use It (2025.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) heavily relies on selecting effective demonstrations to achieve outputs that better align with the expected results. |
| Approach: | They propose a method which integrates a demonstration validation perspective into this field and integrates it into the learning paradigm. |
| Outcome: | The proposed method surpasses all retrieval-based in-context learning techniques across both natural language understanding (NLU) and natural language generation (NLG) tasks. |
Seeing All Sides: Multi-Perspective In-Context Learning for Subjective NLP (2026.findings-eacl)
Copied to clipboard
| Challenge: | Modern language models excel at factual reasoning but struggle with value diversity, authors say . task-sensitive tasks such as hate speech expose this limitation . human disagreement captures the diversity of plausible human perspectives, authors argue . |
| Approach: | They evaluate four large language models with human disagreements on five datasets . they find multi-perspective in-context learning outperforms standard prompting . |
| Outcome: | The proposed approach outperforms standard prompting on English labels while disaggregated soft predictions better align with human judgments in Arabic and Italian datasets. |
Reward Mixology: Crafting Hybrid Signals for Reinforcement Learning Driven In-Context Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for in-context learning (ICL) performance rely on quality and ordering of demonstrations. |
| Approach: | They propose a method that models iterative demonstration selection as a Markov Decision Process and craft hybrid reward signals. |
| Outcome: | The proposed method combines outcome-based accuracy signals with process-oriented signals like stepwise influence and label entropy improvement. |
Representative Demonstration Selection for In-Context Learning with Two-Stage Determinantal Point Process (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods tend to select different demonstrations for each test instance, which is time-consuming and poses limitations in practical scenarios. |
| Approach: | They propose to select a representative subset of in-context demonstrations that can prompt different test instances in a specific task. |
| Outcome: | The proposed method can be used to generate representative in-context demonstrations. |
In-Context Learning with Iterative Demonstration Selection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing literature has highlighted the importance of selecting examples that are diverse or semantically similar to the test sample . Existing studies have shown that the optimal selection dimension, i.e., diversity or similarity, is task-specific. |
| Approach: | They propose to use zero-shot chain-of-thought reasoning to iteratively select examples that are diverse but still strongly correlated with the test sample as ICL demonstrations. |
| Outcome: | The proposed method outperforms existing demonstration selection methods on reasoning, question answering, and topic classification tasks. |
Learning to Select In-Context Demonstration Preferred by Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | In-context learning (ICL) enables large language models to perform tasks with only a few examples as demonstrations. |
| Approach: | They propose a generative preference learning framework that leverages LLM feedback to directly optimize demonstration selection for ICL. |
| Outcome: | Experiments on 19 datasets across 11 task categories show that GenICL achieves superior performance than existing methods in selecting the most effective demonstrations. |
Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching (2025.findings-acl)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) empowers Large Language Models for rapid task adaptation without fine-tuning. |
| Approach: | They propose a method that aligns fine-tuning gradients between entire training set and selected examples to enable in-context learning and fine-uning. |
| Outcome: | The proposed method outperforms random selection on large LLMs from 4-shot to 128-shot scenarios across 9 datasets. |
Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies (2023.findings-emnlp)
Copied to clipboard
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir Radev
| Challenge: | In-context learning (ICL) is a new approach to natural language processing tasks that rely on large language models to make predictions based on context . recent studies have shown that neural symbolic design is the preferred choice for question answering systems because of its limited working memory and unreliable long-term memory. |
| Approach: | They propose to extend in-context learning to question answering tasks that utilize structured knowledge sources and to explore various prompt design strategies for employing LLMs. |
| Outcome: | The proposed approach outperforms the state-of-the-art system by 2.5 points and the best fine-tuned system by 5.1 points on the Spider dataset. |
Many-Shot Scaling of In-Context Learning with Self-Generated Demonstrations (2026.findings-acl)
Copied to clipboard
| Challenge: | In-context learning methods that use self-generated annotations do not scale to many-shot scenarios. |
| Approach: | They propose a framework analogous to semi-supervised learning that uses self-generated annotations instead of ground truth labels. |
| Outcome: | The proposed framework outperforms ground truth ICL under zero-shot, few-shot and many-shot settings. |